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Sir: 

Responsive to the Notice of Appeal deposited May 5, 2004, applicants submit this Appeal 
Brief pursuant to 37 C.F.R. 1.192. Kindly charge the cost of this brief to applicants' assignee's 
deposit account 07-0832. 

I. Real Party in Interest 

The real party in interest is Thomson S.A., who acquired the instant application upon the 
acquisition of Grass Valley Group, Inc, who received an assignment from Tektronix, Inc who 
had initially received an assignment of the application from the inventors Anil Murching et al. 

II. Related Appeals and Interferences 
This is the first appeal in this application. 
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IE Status of Claims 

Claims 1-10 presently remain in the application and currently stand rejected. Therefore, 
claims 1-10 are subject to this appeal. A clean copy of the claims appears in the Appendix. 

IV Status of Amendments 

Applicant initially amended this application by way of a first amendment filed June 2, 
2002. Applicant later amended the application by an amendment transmitted via facsimile on 
July 8, 2002. A further amendment occurred via a paper mailed August 27, 2003. Following 
receipt of a Final Rejection, applicants mailed an Amendment After Final Rejection on March 
29, 2004 in which no amendments were made. 

V. Summary of the Invention 

As recited at page 2 of the English-language specification, applicants provide a method 
for performing Kalman tracking of colored objects within a video image. The method 
commences by separating images within the initial frame based on color. A user provides an 
input selecting an object of interest by identifying a centroid of that object. The selected object is 
tracked through successive frames using a Kalman predictive algorithm applied to the centroid. 

VI Issues 

The issues on appeal are: 

1. Whether the combination of U.S. Patent 6,278,460, issued August 21, 2001, in the 
name of Thomas R. Myers et al., from an application filed December 15, 1998, and U.S. Patent 
5,960,097, issued September 28, 1999, in the name of Carl G. Pfeiffer et al., properly establishes 
a prima facie case of obviousness to render applicant's claims 1-5 invalid under 35 U.S.C. 
103(a); and 

2. Whether the combination of the Myers et al. patent, the Pfeiffer et al. patent, and 
U.S. Patent 5,280,530, issued January 18, 1994, in the name of Timothy I. P. Trew et al., 
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properly establishes a prima facie case of obviousness to render applicant's claims 6-10 invalid 
under 35 U.S.C. 103(a). 

VII Grouping of the Claims 

Claims 1-5 collectively stand rejected* under 35 U.S.C. 103(a) and therefore stand or fall 
together. Claims 6-10 collectively stand rejected under 35 U.S.C. 103(a) and therefore stand or 
fall together 

VIII Argument 

A. Requirements to Establish a Prima Facie Case of Obviousness 

Section 2142 of the Manual of Patent Examining Procedure (MPEP) sets forth the 
criterion required to establish a prima facie case of obviousness in support of a claim rejection 
under 35 U.S.C. 103(a). To establish a prima facie case of obviousness, three basic criteria must 
be met. First, there must be some suggestion of 'motivation, either in the references themselves 
or in the knowledge generally available to one of ordinary skill in the art, to modify the reference 
or to combine reference teachings. Second, there must be a reasonable expectation of success. 
Finally, the prior art reference (or references when combined) must teach or suggest all the claim 
limitations. The teaching or suggestion to make the claimed combination and the reasonable 
expectation of success must both be found in the prior art, and not based on applicant's 
disclosure. In re Vaeck, 947 F. 2d 488, 20 SUP 2d 1438 (Fed Cir. 1991). 

B. The References Relied Upon by the Examiner 
1. U.S. Patent 6,278,460 to Meyers et al. 

The Myers et al. patent concerns a technique for creating a three-dimensional model from 
a plurality of two-dimensional images obtained from a video camera. The method commences 
by obtaining two-dimensional images from the video camera and then using a "ray casting" 
technique to develop a three dimensional image based on the notion of intersecting rays through 
common features of the two-dimensional images. 
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2. U.S. Patent 5,960,097 to Pfeiffer et al. 

The Pfeiffer et al. patent concerns a technique for tracking objects, such as missiles, by 
separating target data from background data. The tracking of an object of interest is initiated by 
storing line of sight vector data while false detections are reduced by assuming the objects follow 
established paths. Observation times are scheduled to observe the object, and a sequence of line- 
of-sight pointing commands are calculated to cause a tracking sensor to be skewed to observe the 
object during the observation intervals. A background adaptive streak detection algorithm is 
used to detect the object. Data from the acquisition and tracking sensors are combined to form 
data tracks while the background adaptive streak detection algorithm parameters are controlled to 
adapt to changing conditions. 

3. U.S. Patent 5,280,530 to Trew et al. 

The Trew et al patent concerns a method of tracking a moving object in a scene by 
forming an initial template of an object and then dividing the template into sub-templates. Each 
sub-template in a successive frame is searched and the displacement to a corresponding sub- 
template in a preceding frame is calculated to enable mapping of the displaced positions to 
produce an updated template. 

C. The Examiner's Rejections 

1. 35 U.S.C. 103(a) Rejection of Claims 1-5 

The examiner contends that the Meyers et al patent discloses a method of performing 
semi-automatic tracking of colored objects within a video image sequence, and teaches the steps 
of (a) separating objects by color and (b) receiving user-provided inputs to select an object of 
interest by identifying a centroid of the object of interest. Since Meyers does not teach tracking 
of the object, the examiner relies on the Pfeiffer et al. patent as teaching tracking of an object 
through multiple frames using a Kalman predictive algorithm. 
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2. 35 U.S.C. 103(a) Rejection of Claims 6-10 

The examiner contends that the combination of Meyers et al. and Pfeiffer et ah, teach 
applying a Kalman predictive algorithm to track the centroid of an object of interest. Given that 
Meyers et al. and Pfeiffer et al. are silent about the color function of the object, the examiner 
relies on the Trew et al. patent to teach the desirability of determining the centroid position based 
on the color function of the object. 

D. Applicant's Response 

1. 35 U.S.C. 103(a) Rejection of Claims 1-5 

In an effort to prove that the Myers et al. patent teaches applicants' separating step, the 
examiner refers to Col. 8, lines 23-45 of that patent which provides: 

"Advantageously, the resolution of the three-dimensional point cloud model may be greater than 
the resolution of the source charge coupled device (CCD) used to create the video stream. Sub- 
voxel data may be interpolated through the inherent nature of how the voxel data is stored—linear 
color gradient interpolation. As subsequent intersecting rays are calculated, the intersection 
geometry is actually a minimal length line between the rays rather than an actual point for the 
majority of the rays. These lines are composed of separate color values for the endpoints of 
the line with the midpoint on the line representing the approximate true intersection point 
of the rays. By dealing with the intersection points as a gradient along a line, sub- voxel 
resolution is generated. The larger number of intersection points calculated yields a higher 
resolution for the model than for the video source. Through this process, a video camera with a 
512 x 384 pixel resolution is able to capture surface texture resolution many orders of magnitude 
larger than each individual frame. For example, 100 frames of 512 x 384 frames (196,608 pixels 
per frame) yields a maximum of 19,660,800*N voxels possible with subpixel improvement of N 
increments. The resolution and subdivision are limited only by the color bit depth of the source 
data." 

The examiner also relies on the disclosure in the Myers patent at Col. 10, lines 32-49 which 
provides: 
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Beginning at block 406, the method determines intersections of rays cast through pixels 
of feature 506 in frames 502 and 504. At block 406, the method compares the quality 
value for the pixel 509 associated with ray 508 with the quality value of pixel 512 
associated with ray 510. At block 408, the method determines whether the quality 
values match within selected bounds. If the quality values match within the selected 
bounds, the method casts rays and calculates the three-dimensional intersection at block 
410. At block 412, the method determines whether the three-dimensional intersection 
falls within selected bounds. If the intersection falls within the selected bounds, then the 
method proceeds to block 414 and writes a data point to the point cloud, e.g., data point 
514. Each data point consists of six fields: the x, y, and z coordinates, and the R, G, and 
B values. These values may be the average of the values for the rays 508 and 5 10 at the 
intersection or some other appropriate representation of each individual data set. 

Neither of these two cited portions of the Myers et al. patent in any way discloses or suggests 
applicants' step of separating objects within a video frame by color. The cited section at Col. 8, 
lines 23-45 of the Myers et al. patent deals with interpolation of volumetric pixel (voxel) data 
using a color linear gradient. Such voxel interpolation has nothing to do with separating objects 
by color, but rather with improving the resolution of a point within the three-dimensional cloud. 
The only mention of color is this cited section of the Meyers et al. patent provides that the lines 
through which the intersecting rays are comprised of color values. However, there is no 
suggestion that the color values are employed for separation purposes. 

The cited portion at Col. 10, lines 32-49 of the Myers et al. patent also provides no 
support for applicants' feature of: "separating objects within an initial frame of the video image 
sequence on the basis of color" At best, this cited portion of the Meyers et al. patent provides a 
description of the "ray casting" process by which points of two-dimensional images are mapped 
into a three-dimensional space. While it is true that the data point written during step 414 in FIG. 
4 of Meyers et al. has individual Red (R), Green (G), and Blue (B) values, the mere fact that such 
data points have color values does not suggest the desirability of separating objects based on 
color. 

Like the Myers et al. patent, the Pfeiffer et al. patent contains no disclosure or suggestion 
of separating objects within a video frame by color. Indeed, the examiner only relies on the 
Meyers et al. patent to purportedly show applicants feature of color separation. Rather, the 
examiner has relied on the Pfeiffer et al. patent for the teaching of Kalman tracking of an object. 

The examiner has failed to establish a prima facie case of obviousness under 35 U.S.C. 
103(a) for claims 1-5 because the examiner has failed to show that the combination of references 
(the Myers et al. and Pfeiffer et al. patents) disclose all of the features of these claims. In this 
regard, the examiner has failed to show that the combination of the Myers et al. and Pfeiffer et al. 
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patents disclose applicants' feature of separating objects by color. Since the examiner has failed 
to establish a prima facie case of obviousness, applicants request reversal of the 35 U.S.C. 103(a) 
rejection of claims 1-5. 

2. The 35 U.S.C. 103(a) Rejection of Claims 6-10 

Applicants' claims 6-10 depend from claim 4 and therefore incorporate by reference the 
feature of separating objects from the background based on color recited in claim 4. None of the 
references, or their combination teaches this feature of claim 4, and therefore, none of the 
references would teach the features of claims. 6-10. As discussed above with respect to the 35 
U.S.C. 103(a) rejection of claims 1-5, neither the. Myers et al. nor Pfeiffer et al. teach applicants' 
step of separating objects from the background based on color as recited in claim 4, and 
incorporated by reference in claims 6-10. The Trew et al patent concerns a method of tracking a 
moving object in a scene by forming an initial template of an object and then dividing the 
template into sub-templates. Each sub-template in a successive frame is searched and the 
displacement to a corresponding sub-template in a preceding frame is calculated to enable 
mapping of the displaced positions to produce an updated template. 

Like the Myers et al. and Pfeiffer et al. patents, the Trew et al. patent contains no 
disclosure or any suggestion of applicants' step of separating objects from the background based 
on color as recited in claim 4, and incorporated by reference in claims 6-10. Indeed, the Trew et 
al. patent says nothing at all about color. 

The examiner has failed to establish a prima facie case of obviousness under 35 U.S.C. 
103(a) for claims 6-10 because the examiner has failed to show that the combination of 
references (the Myers et al. Pfeiffer et al. and Trew et al. patents) disclose all of the features of 
these claims. In this regard, the examiner has failed to show that the combination of the Myers et 
al. Pfeiffer et al., and Trew et al. patents disclose applicants' feature of separating objects from 
the background based on color. Since the examiner has failed to establish a prima facie case of 
obviousness, applicants request reversal of the 35 U.S.C. 103(a) rejection of claims 6-10. 

Applicants' claims 6-10 are patentably distinct over the references of record for another 
reason. Claims-6-10 each incorporate the feature of using a user-selected centroid/reference 
point to first select an object of interest and then tracking that object using a Kalman predictive 
algorithm. As discussed previously, the Trew et al. patent contains no disclosure or suggestions 
of such a feature. At best, the Kalman filter disclosed in Trew is not applied with respect to 
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performing tracking based on a user-selected centroid/object of interest, as recited in claim 4 and 
incorporated by reference in claims 6-10. 

IX Conclusion 

In view of the foregoing arguments, applicant requests that the Board of Patent Appeals 
and Interferences reverse the examiner's rejections. 



Patent Operations 

Thomson Licensing Inc. 

2 Independence Boulevard Suite #200 

Princeton, New Jersey 0854 

June 4, 2004 



I hereby certify that this amendment is being deposited with the United 
States Postal Service as First Class Mail, postage prepaid, in an envelope 
addressed to MS Appeal Brief - Patents, Commissioner for Patents, P.O. Box 
1450, Alexandria, Virginia 22313-1450 on: 



By: Rbbert Bl/Levy 
Reg. No. 28,234 
Phone (609) 734-6820 




CERTIFICATE OF MAILING 




Date 



Lori Klewin 



-8- 



Serial No. 09/3 1 8,682 PU020205 

Art Unit 2625 (6748-US) 
Brief on Appeal 

APPENDIX 



1 1. (previously presented) A method of performing semi-automatic tracking of 

2 colored objects within a video image sequence comprising the steps of: 

3 separating objects within an initial frame of the video image sequence on the basis of 

4 color; 

5 receiving a user-provided input that selects an object of interest from the separated 

6 objects by a user identifying a centroid of the object of interest; and 

7 tracking the object of interest through successive frames of the video 

8 image sequence using a Kalman predictive algorithm applied to the centroid. 

1 2. (original) The method as recited in claim 1 wherein the tracking step comprises the 

2 steps of: 

3 from the initial frame determining a position and velocity for the centroid; 

4 for each successive frame predicting a position of the centroid; 

5 from the predicted position extracting a connected group of blocks that belong to the 

6 object of interest; 

7 measuring the position of the centroid in the successive frame from the connected group 

8 of blocks; and 

9 smoothing the measured position and velocity of the centroid. 

1 3. (original) The method as recited in claim 1 further comprising the steps of: 

2 detecting whether the centroid in the successive frame is within the object of interest and 

3 field of view; and 

4 applying an error recovery scheme to re-identify the object of interest in the successive frame. 

1 4. (previously presented) A method of tracking a colored object moving relative to a 

2 background within a sequence of video image frames, comprising the steps of: 

3 (a) in an initial frame of the sequence, separating objects from the background based on 

4 color; 

5 (b) selecting a separated object by a user identifying a reference point within a boundary 

6 of the separated object T ; and 
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7 (c) tracking the selected object through successive frames of the video image sequence 

8 using a Kalman predictive algorithm applied to the reference point. 

1 5. (previously presented) The method according to claim 4, wherein step (c) includes the 

2 steps of determining the position of a centroid of the selected object and applying the Kalman 

3 predictive algorithm to the centroid. 
1 

1 6. (previously presented) The method according to claim 4, wherein step (c) includes the 

2 steps of determining the position of a centroid based on a color function of the selected object 

3 and applying the Kalman predictive algorithm to the centroid. 

1 7. (previously presented) The method according to claim 4, wherein step (c) includes 

2 the steps of determining the position of a centroid based on luminance of the selected object and 

3 applying the Kalman predictive algorithm to the centroid. 

1 8. (previously presented) The method according to claim 4, wherein each image frame is 

2 resolved into multiple blocks and step (a) comprises the step of segmenting the initial frame 

3 based on color of the blocks. 

1 9. (previously presented) The method according to claim 8, wherein step (b) includes the 

2 step of identifying a color model to which the selected object belongs and step (c) includes the 

3 steps of: 

4 predicting the position of a centroid of the selected object in a subsequent frame T ; 

5 determining whether the predicted position of the centroid in said subsequent frame is 

6 within a boundary of the selected object in said subsequent frame T ; and 

7 in the event that the predicted position of the centroid in said subsequent frame is not 

8 within the boundary of the selected object in said subsequent frame, carrying out a search to 

9 identify a block that belongs to the selected color model. 

1 10. (previously presented) The method according to claim 4, wherein each image frame 

2 is resolved into multiple blocks and step (c) comprises the steps of: 

3 determining position and velocity of a centroid of the selected object in the initial frame T ; 

4 predicting the position of the centroid in a subsequent frame T ; 

-10- 



Serial No. 09/318,682 




PU020205 



% Art Unit 2625 (6748-US) 
Brief on Appeal 

5 from the predicted position of the centroid in said subsequent frame, extracting a 

6 connected group of blocks in said subsequent frame that belong to the selected object?; and 

7 calculating the position of the centroid of the selected object in said subsequent frame from the 

8 connected group of blocks. 
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